Skip to content

feat(insertion-normalization): un-merge CC's join-moves, and stop a re-served entry from re-binding - #295

Draft
Gunther-Schulz wants to merge 51 commits into
cnighswonger:mainfrom
Gunther-Schulz:pr/insertion-join-moves
Draft

feat(insertion-normalization): un-merge CC's join-moves, and stop a re-served entry from re-binding#295
Gunther-Schulz wants to merge 51 commits into
cnighswonger:mainfrom
Gunther-Schulz:pr/insertion-join-moves

Conversation

@Gunther-Schulz

Copy link
Copy Markdown
Contributor

DRAFT — stacks on #272 and #276, review alongside them. This branch is
cut from #272's rewritten head (720ecb4) and merges #276's updated head, because its
tests assert against tools/replay.mjs's gate functions. Base is
upstream/main, so the diff shown by GitHub includes both parents' content;
the material new to this PR is the seven commits after the merge. Same
precedent as #281.

What this is

CC sometimes merges a <system-reminder> message and its immediate
standalone neighbour into a single message
mid-history, then sometimes
un-merges them again a few requests later. Every one of those flips rewrites
history the model has already seen, which busts the prefix cache and re-bills
the whole conversation.

#272 gave the proxy the ability to recognise a message across
re-serializations and to pin the first-seen bytes. This PR closes the two
cases that pinning alone could not:

1. The join-move un-merge. When we recognise that a message CC used to send
standalone has been absorbed into its neighbour, we serve the first-seen
(un-merged) form upstream instead of the newly merged bytes. The join grammar
is a single "\n\n" separator — the same literal the duplicate-suppression
path already keys on — and the probe is byte-exact: the merged wire message's
text must equal pinnedReminderText(predecessor) + "\n\n" + absorbed.text, both
messages must be role: "system", and the absorbed entry's neighbourhood
bounds must resolve on the current wire. Anything else fails closed: no
substitution, raw forward, today's behaviour.

2. The identity fix — a re-served entry leaves the wire-identity space.
This is the part that took a rebuild rather than a patch. Pinned entries are
keyed by (content-hash, role, occurrence-ordinal-within-the-request). A
recognised move keeps the absorbed entry alive in our canonical while CC has
stopped sending it — so its ordinal is a claim about an array it is not in.
The moment a later request carries one more copy of the same recurring text
(measured: a fresh tail reminder taking o=7), the stale entry binds to that
unrelated copy at an inverted position, which both removes it from the dropped
set — so no move recognition can fire — and trips the subsequence check. The
merged message then goes out raw and our bytes flip at an index where CC's
were identical.

The fix marks such an entry rs: true and takes it out of (h, r, o) matching
entirely. Its identity becomes its stored first-seen bytes plus the canonical
slot where we last forwarded them. Each request it gets exactly one of three
dispositions, checked in order: re-fire (the merged form is present again →
re-serve), reclaim (CC flipped back to the original form → clear the mark
and rebind as an ordinary matched entry), lapse (neither form present → the
entry is dropped, never re-served into a region CC no longer carries). Entries
that were never re-served keep absolute (h, r, o) matching byte-for-byte.

Measured — A/B over the live corpus

Two detached worktrees differing only by this diff, replayed over 8.5 GB of
real capture (36 captures, ~10 000 requests) under the serving gate set:

capture before after requests (identical both runs)
s-dc3f8071 2 0 769
s-58c979ce 2 0 2073
s-633915a8 2 0 2630
s-9f9d8a9d 1 0 209
s-0d6f38ba 3 2 1058
corpus total 10 2

Cross-request byte-stability violations go 10 → 2, and the two survivors
are attributed by the gate's own attribution line to a different extension
(deferred-tool-rewrite) and are byte-for-byte the same two pairs in both
trees. Zero insertion-normalization stability violations remain in the whole
corpus.
Safety, conservation, sequence and canonical order read 0 on every
capture in both trees.

Worth stating plainly: the same ordinal collision was firing on four
captures, not the one it was found on. That only became visible because the
measurement was corpus-wide rather than fixture-wide.

Two honesty notes about the sweep as an A/B. The capture count differs (33 vs
36) — three tiny captures (2, 13 and 1 requests) were present only for the
second run, all clean, none in the failing set. And several captures are live
and still growing, so two sequential 8 GB sweeps are confounded in principle —
but the per-capture request counts are identical for every capture in the table,
so for the captures the comparison is about, both trees replayed the same input.

Old-canon compatibility was measured, not argued: tools/verdict-ab.mjs --seed-from-a replays decisions over canon files written by the pre-change
code and is identical across 44 verdict lines / 6 corpora. rs is a new
optional field; canon files from the old code contain none, and under the new
code they take identical decisions. A restart shipping this is
cache-transparent for every existing conversation.

Non-Functional Requirements

  • Size/complexity budget. ~590 added lines in
    proxy/extensions/insertion-normalization.mjs (no new production file, no
    new abstraction, no new env var — it extends classifyPinned,
    resetKeepingPins and findJoinMoves in place). The originating directive
    budgeted 120–200 LOC; the overrun is the reconciliation with feat(insertion-normalization): pin volatile reminder blocks so mid-history rewrites stop busting the cache #272's
    reset-path duplicate suppression, which had to unify two declaration paths
    rather than add a second one. Tests: ~1,170 lines across three files, plus a
    20.6k-line harvested fixture — see the fixture caveat below.
  • Threat model. Conversation fidelity is the protected property; the
    conservation and stability gates are the enforcement. The new risk this
    design introduces is re-serving stored bytes into a context CC has pruned or
    compacted away. The lapse disposition is the mitigation and it fails closed
    — no re-serve — whenever its preconditions are not byte-established on the
    current wire. No new persisted state shape beyond rs, one optional boolean
    on an existing entry.
  • Maintainability. The join grammar stays single-copy (JOIN_SEPARATOR);
    the merged-form probe is the same literal the duplicate suppression already
    uses, seen from the other side.
  • Performance/reliability. The disposition pass is O(reserved entries ×
    neighbourhood) per request; reserved entries measured at 1–2 per conversation
    in every observed instance.
  • Load-bearing? YES. It changes canonical state entries and the bytes
    forwarded on the wire.

Open, and inherited from #272

🤖 Generated with Claude Code

Gunther-Schulz and others added 30 commits August 1, 2026 15:23
…story rewrites stop busting the cache

Claude Code re-serializes <system-reminder> hook blocks inside
otherwise-stable user messages later in the session — moving one into
its own message or merging it into a neighbour — which edits history
mid-prefix and re-bills everything after the edit (reported
independently as anthropics/claude-code#76606; measured here as the
splice/insert-mid class, ~40 kB re-billed per unmitigated hit on a
real session).

The extension keeps a per-conversation canonical model of the message
history keyed by content identity (message-hash.mjs: content hash +
occurrence ordinal — position-independent, so repeated identical
reminders stay distinct). Incoming volatile blocks are pinned to their
first-seen serialization: when CC re-shapes an old reminder, the
forwarded bytes keep the canonical form and the prefix survives. A
history that stops matching the model (compaction, true rewrites)
resets honestly rather than forcing a stale canon — pins survive the
reset, order assumptions do not.

Gated off by default: CACHE_FIX_INSERTION_NORMALIZE=1 enables
normalization, CACHE_FIX_VOLATILE_PIN=1 the pinning. State persists
under the state dir and survives proxy restarts.

Measured on live traffic (513-request session, 2026-07-28): every
observed splice/insert-mid pair forwarded with 0 re-billed bytes; the
canonical-order invariant, cross-request stability and sequence gates
all report 0 violations over 2.5 GB of captures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…lete the pin (anthropics/claude-code#76606)

When CC migrates a hook reminder out of its tool_result into a standalone
system message mid-history, the pin restores the first-seen inline form —
but the migrated copy still forwarded, splicing the same content in twice
(measured live: ~61 kB splice, 124k tokens re-billed on one turn). Now a
standalone message whose wrapper-normalized bytes equal a live pinned
block is suppressed: never forwarded, never given a canonical identity.
Genuine changes (normalized bytes differ) still forward and reset per the
existing rule; assistant-role messages are excluded on principle.
Suppression is re-detected each request from the pin set — no new state
file. One event line per suppression rides the insertion event log.

The real-pair red-green check in the new test file needs the replay
tooling and capture; in this slice it skips, and runs where the tools
land (cnighswonger#276).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
…ture (slice of fork 2dfe0f0)

Path-scoped slice of fork commit 2dfe0f0: only the
insertion-suppression real-pair test and the pinned fixture it falls
back to. The same commit's harvest-pin.test.mjs and
mitigation-output-form.test.mjs changes belong to the verification-tools
slice (cnighswonger#276) and are not part of this PR.

Co-Authored-By: Claude opus-5 <noreply@anthropic.com>
…pinned blocks — the 587k's shape

CC sometimes migrates ALL of a message's volatile blocks out together,
joined into one standalone message (both hook reminders, wrapper-stripped,
joined with "\n\n"), rather than one standalone per block. The existing
single-block suppression set could never match that shape. Each pinned
entry with >=2 volatile blocks now also registers a join-hash — its
blocks' unwrapped texts, in wire order, joined with the one observed
separator — and findSuppressibleDuplicate checks it as a second pass.
No subset-merges, no speculative separators: only the one shape measured
live (capture s-633915a8, msg863/864, and independently confirmed on a
second real occurrence at msg640/641 the same session).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit 78940a0)
…cate is payload (insertion slice of fork e0f8fcb)

Path-scoped slice of fork commit e0f8fcb: the insertion-normalization
tail guard and its tests. The same commit's output-guard
assistant-terminal invariant (proxy/extensions/output-guard.mjs,
test/output-guard.test.mjs) belongs to the output-guard slice (cnighswonger#278) and
is not part of this PR.

Co-Authored-By: Claude opus-5 <noreply@anthropic.com>
… fork da9bf8c

Makes the file slice-portable: without tools/ the real-pair check now
reaches its designed skip instead of dying at module load.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcivCe2iLnKZxpB4qTXzEb
…test reads (fork b1f7c58)

The merge-suppression test does a top-level read of this harvested,
sanitized fixture (16KB, no addresses); without it the file dies at
load. The fork-only exclusion list names only LEDGER-*.json — this
fixture is public on the fork and rides with its test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcivCe2iLnKZxpB4qTXzEb
…story rewrites stop busting the cache

Claude Code re-serializes <system-reminder> hook blocks inside
otherwise-stable user messages later in the session — moving one into
its own message or merging it into a neighbour — which edits history
mid-prefix and re-bills everything after the edit (reported
independently as anthropics/claude-code#76606; measured here as the
splice/insert-mid class, ~40 kB re-billed per unmitigated hit on a
real session).

The extension keeps a per-conversation canonical model of the message
history keyed by content identity (message-hash.mjs: content hash +
occurrence ordinal — position-independent, so repeated identical
reminders stay distinct). Incoming volatile blocks are pinned to their
first-seen serialization: when CC re-shapes an old reminder, the
forwarded bytes keep the canonical form and the prefix survives. A
history that stops matching the model (compaction, true rewrites)
resets honestly rather than forcing a stale canon — pins survive the
reset, order assumptions do not.

Gated off by default: CACHE_FIX_INSERTION_NORMALIZE=1 enables
normalization, CACHE_FIX_VOLATILE_PIN=1 the pinning. State persists
under the state dir and survives proxy restarts.

Measured on live traffic (513-request session, 2026-07-28): every
observed splice/insert-mid pair forwarded with 0 re-billed bytes; the
canonical-order invariant, cross-request stability and sequence gates
all report 0 violations over 2.5 GB of captures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ions via the mid-conversation beta

Every ToolSearch/deferred-tool load makes Claude Code re-send a
different tools[] array. tools[] heads the cache prefix, so each load
re-bills the entire context — the class reported in
anthropics/claude-code#81967 (there triggered by LSP add/remove; the
deferred-tool path hits it far more often). Anthropic's API already
has the fix — the documented mid-conversation-tool-changes-2026-07-01
beta — and Claude Code 2.1.220 ships that beta's documentation in its
own binary without using it on the wire.

The extension freezes tools[] at its first-seen form per session
(keyed session + system-prompt + conversation, so subagents and
sidecars never share a baseline). A mid-session addition keeps the
frozen bytes and instead announces the new tool with a tool_addition
system message anchored at the tail, re-injected at a stable position
on every subsequent request, with the beta header added. Removals and
reorders are held byte-stable outright; a schema change resets
honestly.

Announcements are opt-in per MODEL, with evidence required: the beta
is rolled out per model family and an unsupported one rejects the
whole request with a 400 (measured on claude-sonnet-5 and
claude-haiku-4-5; the haiku error names the gating capability —
mid-conversation system content). An unknown model degrades to
forwarding the changed tools[] — the status-quo bust, never a lost
request — and the first suppressed announcement per model warns with
the way out. tools/probe-tool-addition.mjs measures a candidate model
in one real request; CACHE_FIX_TOOL_ADDITION_EXTRA admits a candidate
on a throwaway proxy for the live probe. Allowlisted with wire
evidence: claude-opus-5, claude-fable-5.

Gated off by default: CACHE_FIX_TOOL_REWRITE=1. Stacked on
pr/insertion-normalization (message-hash identity + system-prompt
sub-key).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…cheduled sweep

Machinery for anyone running this proxy to verify it against their OWN
traffic instead of trusting ours: replay re-runs the real extension
pipeline offline over a capture, per-extension mutation attribution by
hashing between stages, four cross-request invariants (stability,
safety, sequence, canonical order), fidelity against the recorded
forwarded-body hashes (five populations, never one ratio — 0/0 must
not read as "checked and clean"), and --census classification of every
consecutive same-conversation pair, with mitigation gaps priced in
re-billed bytes rather than counted.

harvest promotes structurally novel capture pairs into sanitized,
committable fixtures (novelty judged against per-machine ledgers, so N
machines contribute without duplicating classes). gate-live runs the
replay gate over live captures on a schedule, under the SERVING gate
set read from the running unit — never extension defaults — with heap-
capped children (the cap is a memory-regression check, proven red) and
a status file a checker can consume. cache-sim prices post-pipeline
bytes against raw.

read-lines is the shared pull-based line reader all of them use:
readline's async iterator buffers the whole remaining file once the
consumer awaits (measured 3.27 GB peak on a 1.5 GB capture under code
that called itself streaming); the bite test pins bytesRead against
consumed bytes and went red on line 3 against the readline shape.

docs/dev-loop.md records the working discipline the tools enforce.
Fixtures are sanitized (harvest strips content to structure) and were
audited before publishing. Stacked on pr/deferred-tool-rewrite; pairs
with pr/request-capture, which produces the input format.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…hape watch, hardened sweep

The branch was cut before an afternoon in which the daily sweep and its
own findings changed these tools materially; as opened, the PR shipped
a safety checker with a known bug (the declared-injection exemption
filtered the output side only — an input carrying an injection-shaped
message, e.g. a chained proxy feeding the pipeline its own output, read
as a message drop nobody made; caught by the first stamped sweep and
fixed with a both-ways bite).

What this refresh brings, each with its tests:
- safetyViolation filters declared injections on BOTH sides;
- findSuccessions: conversation-boundary classification (compaction /
  resume-shaped / fork) with opener pricing; interleaves structurally
  suppressed (never-returns + first-appearance conditions, both
  bite-proven — the one-shot-sidecar phantom was caught by its own
  test);
- census edit rows carry lastHumanAt/anchorDelta (the relation that
  attributed the mid-history edit population to reminder anchoring),
  with far-from-anchor rows flagged and their bytes excerpted to local
  stdout only;
- harvest: shape watch counters (dormant thinking classes, baseline
  prefix sizes) and growth-step snapshots (evidence frozen, scrubbed,
  before capture rotation eats it); shape-verdicts.mjs is the consumer
  so the new ledger fields are never orphan telemetry;
- gate-live: --census on every sweep, rows that compared zero pairs
  marked proves-nothing (ok requires a proving row), and the status
  file stamps the proxy/tools source fingerprints it exercised —
  a verdict that names its config but not its code stays "fresh"
  across code changes it never saw.

proxy/source-fingerprint.mjs is included for the stamp; identical file
ships in the capture PR — merges cleanly in either order, same
precedent as the shared fixtures. docs: dev-loop's closing-gate
section and the consumer setup page ride along.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, output-form metric, suppression exemptions

The gate's mitigation metric was input-side only: it trusted the
extension's self-report and never compared what was actually forwarded.
Now each mitigation row carries outputForm (append / splice@N / edit@N),
outputPreserved, and rebilledOutBytes — measured on the forwarded bytes —
which is how a "mitigated" pair that still re-billed 124k was caught.
The census classifies reminder block-migrations (inline <-> standalone)
on splice and edit rows, and the safety and stability checks gain
telemetry-sourced exemptions for the new suppression (a removed message
has no shape to detect after the fact, so exemption keys off the
extension's own suppression records). Extension synced to the cnighswonger#272 tip;
real-pair red-green tests run in this slice, where tooling, extensions,
and capture meet.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
…tput metric; extensions synced

Three replay improvements from operating the gate: an unmissable
stderr warning when a gated capture replays under default gates (the
instrument error that booked a wrong verdict three times in one day —
and whose first live fire caught the operator's own gateless replay);
a --gates-from-capture flag applying the all-boot-records union so
nobody hand-extracts gates; and outputForm now strips cache_control
before comparing (a moved cache marker is not a content splice — five
pairs totalling ~0.6 MB of phantom "re-billed splice" were CC's own
benign marker relocation). Extensions synced to the cnighswonger#272/cnighswonger#273 tips so
the slice's real-capture tests exercise the actual pipeline.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
Real-pair tests (test/insertion-suppression.test.mjs,
test/mitigation-output-form.test.mjs) replay a specific live capture and
SKIP once it rotates out of the retention window. `node tools/harvest.mjs
--pin <key> <n..m>` freezes the sanitized range as a committed,
rotation-immune fixture at test/fixtures/harvested/pinned-<key>-<n>-<m>.json,
reusing the existing scrubRecord sanitizer (never a second scrubber).

Both real-pair tests replay their capture from request 0, not from n,
because insertion-normalization keeps per-conversation canonical state that
only reconstructs correctly if every prior request replayed in order. So
the fixture holds every record (boot, outcome, request) from the start of
the file through request m inclusive, not just n..m -- n..m names the pair
under test, not a truncation point. Stated explicitly in the fixture's
header.

Adds parsePinRange/pinRange/runPin/readPinnedFixture and the --pin CLI
flag; test/harvest-pin.test.mjs covers the mechanism on a tiny synthetic
capture (range parsing, sanitization, CLI end-to-end, and the
readPinnedFixture reader's [n, line] tuple parity with readCapture).
Fallback wiring in the two real-pair tests lands in a follow-up commit.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit da4e7e1)
…nstant

Deviation found while validating the first real --pin (capture s-633915a8,
n=26->28): scrubText collapsed every <system-reminder>-wrapped block to the
literal constant "REDACTED", regardless of its real content. That breaks
insertion-normalization's wrap/unwrap cross-identity check
(findSuppressibleDuplicate/unwrapVolatileText, proxy/extensions/
insertion-normalization.mjs) -- CC sometimes migrates a reminder OUT of its
wrapper into a standalone duplicate message, and suppression fires only
when the wrapped original's stripped bytes hash-match the standalone
duplicate's bytes. Under the fixed constant, the wrapped original always
hashed to "REDACTED" while the unwrapped duplicate hashed its real text
independently, so the two never matched post-scrub.

Measured directly: replaying the sanitized n=26->28 fixture through the
real pipeline gave suppressed=0 / outputForm="splice@31" (the pre-fix
defect shape) instead of the live capture's suppressed=1 /
outputForm="append". The committed pinned fixture would have silently
reproduced stale, wrong behaviour once the live capture rotated away --
worse than the SKIP it was meant to replace.

Fix: a wrapped reminder now re-wraps scrubText's own recursive token for
its inner text instead of a fixed placeholder. The wrapper tags still
survive verbatim (any check that only tests for wrapper PRESENCE is
unaffected -- verified against test/harvest.test.mjs's existing
"wrappers survive" test, unchanged and still green), and two reminders
with equal real bytes -- wrapped or not -- now hash equal after scrubbing,
matching what the sanitizer already guarantees for ordinary text.

Verified: test/harvest.test.mjs (14/14, no regression), and the fixed
scrubber's output re-run through the actual pipeline (findMitigationGaps,
findSafetyViolations) reproduces the live capture's real values exactly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit fda83cc)
…eating the evidence

Wires the fixture-fallback into both real-pair tests
(test/insertion-suppression.test.mjs, test/mitigation-output-form.test.mjs):
live capture present -> unchanged path (readCapture); capture absent ->
fall back to the pinned fixture via readPinnedFixture if one exists; both
absent -> skip with a stated reason, as before. Both readers yield the same
[n, line] tuple shape, so the replay loop in each test is unchanged either
way. Both the capture path and the fixture path are overridable via
CACHE_FIX_TEST_CAPTURE_OVERRIDE / CACHE_FIX_TEST_FIXTURE_OVERRIDE so the
fallback's own red-green test never has to touch the real capture file,
which is read-only evidence shared with other concurrent work.

Commits the first real pin: test/fixtures/harvested/pinned-s-633915a8-26-28
.json, the n=26->28 pair both real-pair tests already reference in their
comments, produced with `node tools/harvest.mjs --pin
s-633915a8 26..28` (holds the full prefix 0..28
per the previous commit's replay-from-start finding) and verified against
the fixed scrubber from the prior commit: replaying it through the actual
pipeline reproduces the live capture's exact values (mitigated=true,
outputForm="append", suppressed=1 at index 31, 0 safety violations).

test/harvest-pin.test.mjs gains the fallback's red-green proof: both
real-pair test files are run as actual subprocesses (not re-derived) with
env overrides -- capture+fixture both absent skips (never a false pass),
capture absent + the real committed fixture present runs and PASSES
(never a false fail). Both real-pair suites are green in both modes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit 2dfe0f0)
… + tail guard

Brings proxy/extensions/insertion-normalization.mjs and its real-pair
test to the cnighswonger#272 slice tip (fork e0f8fcb): the merged-standalone
join-hash set and the tail-position suppression guard. The extension
rides here so this slice's tools replay the same behaviour cnighswonger#272 ships.

Co-Authored-By: Claude opus-5 <noreply@anthropic.com>
…r path (slice of fork 2a11487)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TcivCe2iLnKZxpB4qTXzEb
…nswers daily

findDuplicateRequests classifies every ADJACENT same-conversation pair
whose incoming message array is byte-identical (same length, same
per-message hash at every index) — the "hidden duplicate request"
falsifier (CC#78420) that was previously a throwaway python scan, now a
census kind that rides every --census run (BACKLOG "Duplicate-request
probe -> census check (Q1)"). Rides both JSON and text output alongside
the existing toolsDeltas/edits/blockMigrations census kinds.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit 7bc3462)
…, log on staleness

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit 083c5d6)
… set

Out-of-band runs (operator shell, doctor) read gates from an env that
never carries them, so every gate-dependent verdict warned "off" on a
serving machine. Env-set still wins; unset falls back to
cache-fix-gate-status.json's recorded gates. Includes the import the
first draft's own catch was masking (a ReferenceError swallowed into
"no status file" — the fail-open hid the failure).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
(cherry picked from commit 8e91c69)
…ts on telemetry, never on shape

fresh-session-sort's relocate branch prepends content to messages[0] on a
block type's first appearance in the array — a deliberate one-time bust
(the #34629 class) that reads to replay's cross-request stability check as
a self-inflicted byte flip (n=2024->2025 in s-58c979ce: CC diverges at
index 1, our output diverges earlier, at index 0).

The extension now reports what it did (ctx.meta.freshSessionSortStats:
relocated block types + a firstAppearance flag computed from the SAME
backward scan that already finds each type's latest instance, plus the
target message index). replay's stability check gains a telemetry-keyed
exemption mirroring suppressedIndices' discipline: exempt only when the
CURRENT entry's telemetry names the exact outDiv AND at least one relocated
block is a genuine first appearance — never re-derived from divergence
shape alone, so a relocation reported without telemetry, or reported as a
recurrence, stays a violation. Exempted entries are reported separately
(findStabilityExemptions, --json output, console section) rather than
silently dropped.

Verified: unit bites both ways (with telemetry -> exempt; without, or with
firstAppearance:false -> still a violation), red-before-green on both the
extension telemetry and the checker exemption. Real-pair replay on
s-58c979ce confirms PRE: exactly 1 stability violation at n=2024->2025,
attributed fresh-session-sort (measured independently twice). POST:
0 violations, 1 exemption annotated, safety/sequence/order violations
unchanged at 0 (confirmed by the dispatcher's run on this working tree).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit e41e068)
…hared subset

forwardedStable compares the whole forwarded tools[] array, so a genuine
new-tool announcement always reads as "unstable" even when every tool CC
already knew about round-tripped byte-identical. heldStable narrows the
claim to the SHARED-name subset of a pair's forwarded tools (the tools
present on both sides), the guarantee deferred-tool-rewrite actually makes.
gate-live's toolsDeltas summary carries it beside forwardedStable.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
(cherry picked from commit 813edc8)
…p — born with its reader

Also pins the gate's real enable value in place: "on", not "1" — the
booked =1 flip would have silently left the extension off.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
(cherry picked from commit 9876fff)
…oded count bit its author

The sixth-row commit shipped red because the suite pipe reported
grep's exit, not the test's; the two enumerations (count 8, five-name
list) were exactly the corpus's hardcoded-count anti-pattern, now
table-derived so a legitimate row cannot redden them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TZxGrF1LRBvmb7cFXmS2DH
(cherry picked from commit 7790dff)
…e refuted

The BACKLOG item this fixture was harvested for ("per-block standalone
match") assumed the flap's two standalone legs were two per-block copies
of one pinned reminder, which pinnedBlockHashes could not match. Measured
against the real bytes (capture s-0d6f38ba, n=102/104/105/108), all three
parts of that premise are false:

  - pinnedBlockHashes ALREADY registers each volatile block's
    individually-unwrapped text hash — it has since the original #76606
    suppression, before the join-hash sibling (78940a0);
  - the standalone legs are not per-block copies. msg86 is the JOIN of
    msg85's four reminder blocks ("\n\n", 1256 chars) — which the
    join-hash already matches — and msg94 is msg92's single reminder
    (683 chars) — which the per-block hash already matches. Executed:
    findSuppressibleDuplicate returns a hash for BOTH;
  - the leg that matches nothing is msg91: a CROSS-MESSAGE join, msg89's
    unwrapped reminder + "\n\n" + the whole standalone system message
    that followed it at msg90 (1106 chars). No hash set spans two source
    messages, and 78940a0's guard (c) refuses cross-entry joins by
    design.

So suppression is not where the flap escapes. classifyPinned returns
reset("edit-shaped") at the isEdit co-location test BEFORE the
suppression pass runs — msg91 lands in the gap left by dropped msg90,
which is exactly the shape that test calls a genuine edit. Confirmed by
the shipped pipeline itself, not only offline: replay under the
capture's own boot gates (10/10) logs n=104 reason=edit-shaped, and the
census still lists all three flap pairs.

The fix therefore needs a design decision above a build brief — whether
a cross-message join may suppress at all (suppressing msg91 would drop
msg90's bytes from the wire entirely, since its canonical entry is
dropped and this extension never re-adds a message) and whether
suppression may run before the edit-shaped reset. No code change here.

The fixture is the evidence, taken before the capture rotates: full
message arrays for all four requests of the three flap pairs, so both
the suppression relations and the reset decision reproduce offline.
Sanitized via harvest's scrubMessage, EXCEPT the hook-reminder texts
that participate in the migration, kept raw — scrubText is not a
homomorphism over concatenation, so a merged standalone would scrub to
a token unrelated to its parts and the join relation, the point of the
fixture, would not survive. Every raw text was read in full first: all
are harness-generated hook reminders, no operator prose and no
host/network identifiers. Fidelity checked against the raw bytes: same
action, same reset reason, same per-message suppression verdicts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 090a110)
…quests is flagged, not read off adjacent lines

The 2026-07-30 221k event (threat matrix row 4, session 0d6f38ba,
n=102->104->105->108 in 11 seconds) was visible only by reading three
adjacent census lines and noticing the direction column alternate. A
one-way migration is absorbable by the volatile pin; an oscillation is
not, when the pin classifies only one of the two shapes — it busts on
every second flip at best. That distinction now has a name in the
census instead of living in a reader's attention.

blockMigration rows carry the migrating block's `hash` (the unit hash
scanBlockMigrations already computed — no second notion of sameness),
and the flap scan runs per conversation group, so the window counts
requests of that CONVERSATION, not of the wire: cache prefixes are
per-conversation, and a co-tenant's traffic is not part of the clock.

Red-first, three mutations each biting its own test: neutralised
markFlaps -> the two BITEs and the boundary case go red; window bound
removed -> the 6-requests-later guard goes red; block-hash identity
dropped -> the different-blocks guard goes red.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit fc44da3)
…e that shed a sibling is not a standalone emergence

Measured on the real flap bytes (fixture flap-s-0d6f38ba-86.json, pair
n=102->104):

  PREV[92] user [tool_result, text(<system-reminder> 720 chars)]
  CUR [93] user [tool_result]        <- PREV[92] having SHED its reminder
  CUR [94] system "…" (683 chars)    <- PREV[92]'s reminder, unwrapped

Both guards of the existing definition were satisfied by a block that
never moved. Two messages were inserted above, so the host's own index
moved and the same-position guard compared against an unrelated message;
and `standalone` is `blocks.length === 1`, which a message that SHRANK to
one block satisfies. The tool_result was therefore reported as migrating
92->93 while sitting exactly where it always was.

The census reported 6 migrations on that capture where 3 exist, and after
the flap annotation each phantom also carried a `flap` tag — telling the
reader that two blocks oscillate when one does.

Candidacy condition restored from the class the section already names
(the reminder swap): a block is a migration candidate only where it
appears <system-reminder>-WRAPPED on its INLINE side — as the source unit
when leaving a multi-block message, as the destination unit when joining
one. This narrows the check to its own declared subject rather than
adding a new rule.

Note this is NOT the shrink predicate inside blockUnits: whether a
message shrank is not knowable from the message alone, and pair-locally
it is not knowable at all for the reverse direction — in pair
n=104->105 the one-block msg93 is simply a one-block message, so a
shrink-based rule leaves the standalone->inline phantom standing. The
wrapper is pair-local and definitional.

Red-first, and probed for over-narrowing rather than only for the fix:
all three bites go red with the condition removed — the fixture bite
with "6 !== 3" on the real bytes — and green with it; the documented-real
n=26->28 migration (30->31) survives untouched on s-633915a8, as does the
second flap there (105->107->108->109), and only one further phantom
drops (n=1515->1528). Gate 0/0/0/0 on both captures under their own boot
gates.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 47defba)
…, or red

The four existing gates are all positional: did our bytes move earlier than
CC's, did we change the message sequence, did a normalize get followed by a
reset, does canonical order track the wire. None of them can see a message CC
sent that we never forwarded and whose content exists nowhere else, because a
deletion leaving the survivors positionally consistent is invisible to all
four. pin-and-suppress (#76606 decision B) deletes messages on purpose, and
the mitigation this gate is a precondition for deletes one more.

DEFINITION, per request, CC's raw array R against the forwarded array F.
R-side: every content unit of a non-assistant message is forwarded
byte-identically, or belongs to a DECLARED suppression (stats.suppressions,
the extension's own report) whose bytes are reconstructible from F — as a
forwarded unit, as the "\n\n" join of one message's reminder blocks
(78940a0's merged standalone), or as the CROSS-MESSAGE join of two adjacent
forwarded messages — or is a declared strip. F-side: every unit is present in
R, or was sent by CC in an EARLIER request of the same conversation — which is
what "the pin forwards the FIRST-SEEN bytes" means, stated as a checkable
property instead of trusted — or is a declared tool_addition injection.

Population is non-assistant messages, definitionally rather than
conveniently: every mechanism that can delete or re-serve content here is
confined to it (classifyPinned skips assistant entries before suppressing;
pinnedForwardForm passes anything but a user entry through untouched).
Measured over 936 requests of four live captures, the only blocks the
pipeline does not conserve are assistant tool_use (rewritten in place by
tool-input-normalize) and assistant thinking (sanitized) — a separately-gated
class with no telemetry to key an exemption on, so including it would fire on
two declared behaviours. The count of skipped blocks is reported rather than
hidden, so the boundary is visible to a reader of the row.

THE EXEMPTION REGISTRY IS NOT A GUESS — the gate populated it by going red.
Its first sweep over the live corpus was clean on 29 of 30 captures and
reported 645 violations on s-633915a8, all of kind `lost`, all at message 0.
Stage-by-stage replay of request 822 named the cause: RAW 6 units, after
fresh-session-sort 3, the three removed being a /compact
`<local-command-caveat>`, its `<command-name>` and its
`<local-command-stdout>` — the harness quoting its own slash command back,
which that extension deletes on purpose (fresh-session-sort.mjs, "Strip
/clear artifacts from first user message"). Declared behaviour, not lost
conversation, so it is exempt via that extension's own exported predicate
rather than a restatement of it. Ruling the instrument out before filing the
defect is what kept this from being reported as a content-loss bug in
fresh-session-sort.

Red-first, seven mutations each biting its own test: R-side loss detection
neutralised -> the two loss bites; suppression reconstruction neutralised ->
both suppressed-without-copy bites; F-side invention detection neutralised ->
both invention bites; the first-seen registry made global instead of
per-conversation -> the co-tenant bite; join reconstruction dropped, cross-
join dropped, cross-join adjacency dropped -> their three tests respectively.

THE REAL RED, on the committed flap bytes rather than a synthetic shape: a
throwaway variant applying naive merged-standalone suppression over fixture
flap-s-0d6f38ba-86.json (suppress the three migration arrivals, re-serve
nothing) reports

  n=104 suppressed-without-copy: in[91] (system): 1 of 1 unit(s)
        reconstructible from neither a forwarded block nor a forwarded join
  n=108 (same)

while msg86 and msg94 pass — their copies really are on the wire. msg91 is
the CROSS-MESSAGE join, and its second constituent is msg90, whose bytes
naive suppression drops from the conversation entirely. That is the
evidentiary answer to the parked design question, and the shipped pipeline is
green on the same four requests.

Rides gate-live as a row field with the same rank as safety, and fails the
sweep on its own.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
(cherry picked from commit 95ca0cb)
Gunther-Schulz and others added 21 commits August 1, 2026 15:42
…Of; amend the hand-rolled-identity rule

The row-4 canonical rule was hand-derived from two occurrences in one
capture: it reproduced one byte-exactly and failed the other. That split is
invisible at n=1 and is the difference between a mitigation that absorbs a
bust and one that moves it, so the design could not proceed on it. The
hand-derivation was the prototype; this is the mechanism.

tools/reminder-migration-census.mjs byte-tests the canonical rule across a
capture corpus and reports four verdicts, deliberately separated so none can
inflate another:
  EXACT     reconstruction is byte-identical to CC's own later message
  EXTENDED  CC's later form carries NEW text — a different class entirely
  DROPPED   blocks vanished with no counterpart — rule never exercised
  MISMATCH  a real hole; any occurrence blocks shipping

Result over the corpus (22 captures, 74 conversations, 3431 pairs):
24 EXACT (85.7%), 3 EXTENDED, 1 DROPPED, 0 MISMATCH. The rule holds on every
occurrence it applies to; the EXTENDED cases are new-information arrivals and
stay booked as their own class.

Getting there took three corrections to this tool, all one family — it
hand-rolled conversation identity instead of importing it:
  1. compared before[i] to after[i] by INDEX; one inserted message shifts
     every later index
  2. paired ADJACENT capture lines; interleaved tenants put two requests of
     one conversation several lines apart (replay.mjs already documents this)
  3. scored DROPPED blocks as rule failures, manufacturing a blocker
The first two together reported 475 failures / 99.3%, every row reading
actual=0ch — the tell that no counterpart was found at all. Corrected: 0.

So replay.mjs now EXPORTS conversationOf with the rule stated at its
definition: any tool comparing two requests must group by conversation, never
by capture adjacency and never by index. Exporting beats restating — a second
tool re-deriving the identity is how the two drift.

docs/dev-loop.md's "Never hand-roll identity in a probe" is AMENDED rather
than joined by a second section: the new instance is added (a tool, not a
throwaway probe, which is what let it look authoritative), and two corollaries
it forced are stated — extend an existing tool before writing a new one,
because reuse inherits hard-won correctness while a new file re-earns it from
zero; and group by conversation, exporting an identity rather than restating
it when one is missing.

Verified: --selftest green (canonical join, unwrapped passthrough, the three
classify verdicts incl. EXTENDED-is-not-EXACT, host detection); full-corpus
run reproduces the numbers above; replay.mjs still imports cleanly with the
new export.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HDqQZZCtpbLrSQCd68dpgq
(cherry picked from commit 7b2a5ef)
…d verdict

On 2026-07-31 a single bust took a six-step hand investigation — statusline,
worktime ledger, CC transcript, proxy journal, capture pair, body diff —
before replay --census could even be pointed at it. Six of the ten steps were
manual, and the two most valuable findings came out of steps nobody repeats
under time pressure. One of them was an entirely uncovered bust class, found
only because a diff happened to be read: nothing in the stack had a "none of
the above" branch, so a class with no matrix row was invisible by
construction.

bust-triage chains the existing tools rather than reimplementing them
(dev-loop.md, "Never hand-roll identity in a probe"): classification is
replay.mjs's censusPair, the container byte-test is
reminder-migration-census's canonical/classify, and only the
ledger/transcript reconciliation and the matrix lookup are new here.

THREE answers, never two:
  MITIGATED     known class, matrix row not open
  KNOWN-OPEN    known class, row N, still open — prints that row's status
  UNCLASSIFIED  no row matches; an unrecognised class is the payload
  UNVERIFIABLE  a step could not run; never folded into a pass

It also reconciles the ledger against the transcript, because those two
disagreed live today — the display upgraded a raced cause while the record
kept "other", and the divergence was invisible until something compared them.

Three pair-selection defects were found and fixed by running it against a
bust whose answer was already known by hand:
  1. +30s slack selected a request 35s AFTER the bust — an append-only pair
     that reported UNCLASSIFIED, a phantom new class. The busting request
     always PRECEDES the ledger stamp; the hook runs after the response.
  2. no sidecar guard: one session id spans the main thread, subagents and
     1-message bootstrap calls, so the newest request before the stamp was
     often a sidecar. A 44k rewrite classified as "identical" on n=1->n=1.
  3. (in the census it imports) conversation grouping rather than adjacency.
Each produced a confident wrong answer that looked like a finding.

Verified: --selftest green (classToRow invents no row for unknown classes,
retraction and hit-cause handling on a synthetic ledger, matrixRow reads row
4 as OPEN). End-to-end against the known bust it reproduces the hand-derived
result exactly — transcript messages_changed/105006, pair 11:40:45->11:41:05
n=130->124, census replace/edit, row-4 container migration at host 97 (EXACT),
verdict KNOWN-OPEN. A bust whose capture has rotated away reports
UNVERIFIABLE rather than guessing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HDqQZZCtpbLrSQCd68dpgq
(cherry picked from commit a774176)
Pre-compact work that ran today but was never committed: hostId
(tool_use_id of the leading tool_result) locates where a host landed in
the after-request; a standalone candidate must sit AFTER its host —
content matching alone picked an identical system message hundreds of
slots away (offsets like -839). EXACT findings now carry the offset and
the summary prints a placement tally, since emitting the right bytes at
the wrong index diverges the prefix just the same. Verified by
execution: today's runs over the 77fe2779 capture used this code
(placement: +1, single placement).

Co-Authored-By: Claude Fable 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HDqQZZCtpbLrSQCd68dpgq
(cherry picked from commit 3afce21)
The census byte-gate slurped each capture with readFileSync and swallowed
the failure, so on the live corpus the four LARGEST captures — 6.2 GB of
7.9 GB — fell out of every verdict it ever produced while it reported "25
capture(s)" and exited 0. Same RangeError replay.mjs was fixed for on
2026-07-28, re-committed in a newer tool, plus the three-answer violation:
an absence reported as a pass.

The read now shares read-lines.mjs with the gate, grouping keeps only each
conversation's PREVIOUS request rather than the whole file, and a run that
could not read something names it in the header, in a COULD NOT READ block,
in --json, and in the verdict block — and exits 1.

Measured over ~/.claude/cache-fix-captures/*.jsonl:
  before  25 capture(s), 82 conversations, 3646 pairs   (4 skipped SILENTLY)
  after   read 39/39, 0 UNREADABLE, 165 conversations, 10290 pairs
  tallies on the previously-readable files unchanged: 17 EXACT / 10
  EXTENDED / 1 DROPPED / 0 MISMATCH (baseline re-run on the same 35 files
  agrees at 3662 pairs vs 3661 — live captures grew between the two runs)
  2.4 GB capture censused in 6 s under --max-old-space-size=512

FINDING from the newly-readable 79%: placement is no longer single. 55
EXACT at host+1 and 3 at host+4, so the tool now prints MORE THAN ONE
PLACEMENT where it used to print "single placement; safe to emit" — a
normalization emitting at host+1 would diverge the prefix on those 3.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit a77c930)
EXTENDED was one label for two phenomena, and only one of them is new
information. The remainder beyond the canonical reconstruction is either a
standalone role:"system" message the PREDECESSOR already sent (CC merged an
existing message into the migrated one — nothing new crossed the wire) or
content no earlier request carried. That distinction decides a mitigation,
had been hand-derived once (extended-absorb-report §b1) and lived nowhere,
so the next session would have derived it again.

The header's "NOT absorbable by any normalization — new information, not
re-serialization" is corrected: refuted for the merged sub-class, and the
correction is measured rather than argued.

Measured over ~/.claude/cache-fix-captures/*.jsonl (39/39 read):
  21 EXTENDED — 21 MERGED-STANDALONE, 0 NEW-TEXT
  the report's 9 known occurrences each reproduce as MERGED-STANDALONE
  extendedSub rides --json, so bust-triage can key on it

Sub-verdicts come from subclassifyExtended, checked against the BEFORE
request's standalones only: matching the after request's own would make
every merge trivially true, since the classified message is one of them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit a301ef1)
Row 22's refutation came from a throwaway drop-scan probe with hand-rolled
per-message hashes — the tell that a check was missing. It is a check now:
every same-conversation pair whose message count DECREASED is classified
PURE-TAIL-PRUNE / INTERIOR-DIVERGENT / UNANCHORED, with the re-billed
suffix length on every row, using firstDivergence and isHumanTurn imported
from replay.mjs rather than restated.

gate-live runs the census as a second child per capture under the same heap
cap, so the migration byte-test and the prune summary land in the daily
sweep and in cache-fix-gate-status.json. Coverage failures bite (a capture
the byte-gate could not read makes the row not clean — that is item 1's
done-criterion); findings (MISMATCH, interior prunes) are carried, never
failed, since they are facts about CC's traffic and a check that fires on
non-defects trains its reader to ignore red.

DEVIATION from the backlog entry's verifier, with its basis. The entry
predicts 12 events, 10 pure / 2 interior on s-77fe2779. Events reproduce
exactly (12) and 11:41:05 reproduces as INTERIOR-DIVERGENT (breaks at 97,
anchor 123, re-bills 27 of 124). 11:31:58 does NOT: read at the bytes it is
the same phenomenon as the ten pure ones — CC pruned a [SUGGESTION MODE: …]
scaffolding block and the user's real turn landed at the same index — and
differs only in the live turn having produced 3 messages instead of 1-2.
Splitting it off requires a "within N of the tail" threshold that no
definition produces, so the boundary here is the ANCHOR (isHumanTurn), the
same relation row 4's verdict rests on. Result: 11 pure / 1 interior.

Corpus-wide, 39/39 captures: 226 drop events, 181 pure, 45 interior, 0
unanchored. Two interior events re-bill nearly everything — 2026-07-31
12:42:11 (n=688->675, breaks at 4, re-bills 671) and 11:40:24 (n=83->81,
breaks at 4, re-bills 77); the first lives in a capture that was unreadable
until the previous commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 404d5fc)
bust-triage read only k:"hit", so on 2026-07-31 the statusline showed
`❄ 55k compact (8m)` (ledger k:"cost", t=1785505434) while --list showed
nothing newer than 90 minutes earlier and the default run silently triaged
an older, unrelated event. An event the operator can SEE must never be
missing from the tool that explains events.

coldEvents() now reads the whole ❄-visible population and splits it: "bust"
(k:"hit", triageable) vs "controlled" (k:"cost" plus legacy k:"resume" — a
cost the operator or the auto-compact ceiling caused). busts() keeps its
old meaning, so nothing downstream shifts. --list labels controlled events
CONTROLLED(<cause>), and when the newest cold event is controlled the
default run states "cannot triage: controlled cause" and names the bust it
fell back to, in text and in --json.

The controlled set comes from claude-worktime itself, not from this tool:
the ❄ token advances on cold_hit and cold_cost, and its `--cold --all`
filter lists hit, cost and legacy resume. That is why resume is included
where the backlog entry named only cost — the done-criterion is that a
❄-visible event can never be absent, and 3 resume records are in the live
ledger.

Verified against the live ledger: --list carries
"2026-07-31 13:43:54  55k  CONTROLLED(compact)  77fe2779", and a no-args
run over a copy truncated to that instant prints the NOTE and falls back to
the 12:25:23 bust — the same substitution that used to happen in silence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 6efce90)
…itization

scrubText tokenized whole texts, so scrub(a + "\n\n" + b) !=
scrub(a) + "\n\n" + scrub(b) — measured, not inferred
(extended-absorb-report §c5). The prefix and join relations that DEFINE
EXTENDED and the merged-standalone shape died at scrub time, so a fixture
pinned for that class could not reproduce the class it was pinned for and
extended-absorb.test.mjs had to hand-build synthetic tokens instead.

The scrub now splits on "\n\n" — the domain's join, the same literal the
census's canonical()/classify() and insertion-normalization's duplicate
suppression already hardcode — tokenizes each segment, and rejoins. Wrap
handling runs first and unchanged, so the fixed-constant lesson holds at
paragraph granularity. Inputs outside the join contract degrade to the old
whole-text behaviour: no crash, no leak, relation simply not promised.

The accepted privacy delta is metadata only — paragraph count,
per-paragraph lengths, cross-text sharing of identical paragraphs, never
content bytes — accepted by operator ruling for this local, controlled
deployment, with the audience caveat carried in the scrubber's comment for
anyone harvesting third-party traffic.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit bffcb05)
a301ef1's test file said a harvested fixture cannot reproduce the merged-
standalone class, citing the then-PARKED §c5 (scrub tokenized each text
independently, so the prefix/join relation did not survive). bffcb05 landed
a "\n\n"-homomorphic scrub while these items were in flight, so the premise
is dead and the comment would have taught the next reader the opposite of
what the code does.

Verified against the shipped scrubMessage rather than from the commit
message: prefix and join relations both survive, classify() returns
EXTENDED on the scrubbed bytes and subclassifyExtended returns
MERGED-STANDALONE. The fixtures stay synthetic — a unit test wants a
minimal pair it controls — but that is now a preference, not a constraint.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 496fbf0)
…ow (verification slice of fork a1170a7)

Only the tools/ half of fork a1170a7 travels in this slice: replay's
stability and conservation checks learn the join-move action so a
re-served move reads as designed behaviour, not as a violation. The
extension half and its tests ride in the insertion slice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit a1170a7, tools/ paths only)
…, diffed

Graduates the throwaway A/B script of the unit-2b build (closing report
2026-07-30, "Corpus A/B — nothing else moved") because the reserved-entry
identity build needed it a second time: a probe used twice graduates or dies
(dev-loop). It answers a question replay.mjs is single-tree by construction and
cannot ask — does CHANGING the code change any decision it takes on the
committed corpus — by holding two extension modules resident at once and
diffing the verdict line (action, reset reason, pinned, suppressed, moved,
dropped, forwarded length) per request.

Two modes, two different questions. Independent chains asks whether
steady-state behaviour moved. `--seed-from-a` feeds tree B, at every request,
the canonical tree A wrote for the preceding one — the OLD-CANON COMPATIBILITY
probe, i.e. whether a restart is transparent for conversations already in
flight, which is threat-matrix row 3's question and previously answerable only
by argument.

Three things carried over from the lessons that produced it. It exits 2 with
COULD NOT VERIFY when no fixture yields a replayable request, because the first
version of the unit-2b probe printed "IDENTICAL" over two empty dumps after
crashing on both trees — demonstrated red here against an empty fixture
directory, not asserted. It reads all THREE committed fixture shapes
(`{requests}`, `{header, records}`, `.jsonl` capture records) and names every
file it skipped, because the first draft of the reader silently saw 2 of the 6
message-array corpora and would have reported a 9-line "IDENTICAL" as a
44-line one. And it groups by the extension's OWN `resolveInsertionSessionKey`
rather than chaining one canonical per file — the pinned fixture alone carries
six conversations, and comparing across them is the hand-rolled-identity error
this repo has paid for four times.

Trees are given as git refs (checked out detached into a scratch worktree and
removed afterwards) or as directories; the shared working tree is never used
as a scratch checkout.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit da8b837)
…ery, and the absence scan — match fork 687cbc5/eb4f844

The slice ships the harvester, so it ships the FIXED harvester:
scrubBlock recurses into source (the payload one level below where the
old scrubber looked — the measured five-PNG leak class) and fails
closed on any long string there. Capture discovery in the two
real-pair tests recovers the file by hashing candidates against the
fixture's own token instead of hardcoding a capture id (a capture
UUID plus a home path is a live identifier in a public tree).
tools/absence-scan.mjs + its test make sanitization CHECKED rather
than claimed, per the cnighswonger#272 fixture-strategy thread; one allowlist
entry added with provenance (upstream's own org_id example in
docs/directives/proxy-cache-warmer-v3.7.0.md). The grafted nesting
tests pin the source.data class red-first at unit level.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016y33RMV399iYMXFEbAfQCk
… import

# Conflicts:
#	test/insertion-suppression.test.mjs
…he RESET path too

The suppression built for #76606 (decision B) was silently disabled by any
reset. `resetKeepingPins` restores the pins and returns BEFORE the
migrated-duplicate pass, so CC's standalone copy of an already-pinned reminder
went out on the wire beside the restored inline form — the reminder forwarded
twice, mid-array, moving the cache's longest-identical-prefix boundary to just
before it.

Measured live 2026-07-31 (session 77fe2779, request 11:41:05.778Z): the
telemetry for that exact request reads
  action=reset resetReason=not-subsequence pinned=2 suppressed=0
Pins restored, suppression skipped. Outcome: edit@98 of 123, transcript
cache_miss_reason messages_changed / cache_missed_input_tokens 105006, ~104 kB
re-billed.

Scale, from this file's own measurement: 125 resets across 350 requests,
roughly one request in three. A mitigation that switches off on a third of
requests, without saying so, reads as shipped and behaves as absent.

The fix reuses what the reset path already holds — the pins it just restored,
and priorCanonical — rather than adding a mechanism: a standalone duplicate is
suppressible exactly when the inline form it duplicates is being restored.
Suppressed entries are removed from the forwarded array and excluded from the
canonical, preserving the success path's invariant that the canonical
describes the wire we just forwarded.

NOT claimed: what made the survivors non-subsequence on the live request.
`not-subsequence` requires matched entries to invert in order, which a plain
pruning does not produce, so the "the suggestion-mode pruning caused it" story
is unverified and is stated as such in the test header. The fix does not depend
on the trigger.

Verified — and two false starts are recorded because both nearly shipped a
claim: (1) the first regression scenario never reached the reset path at all
(it classified as "normalized"), so it proved nothing until rebuilt around an
explicit reorder that yields resetReason="not-subsequence"; (2) the first
instrument proof reverted the code into a syntax error, and a red from a file
that will not load proves nothing — redone by neutralizing only the suppression
with the module still loading. With it neutralized the two regression tests go
red and the three guard tests stay green; restored, 5/5 here and 73/73 across
all four insertion suites.

Guards against the opposite failure: a reset with nothing to suppress is
unchanged, a standalone matching no pinned block is forwarded untouched, and a
tail-position duplicate is never suppressed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HDqQZZCtpbLrSQCd68dpgq
(cherry picked from commit 059aae3)
…the EXTENDED-absorb design that measurement killed (insertion slice of fork 5c4d70a)

The reset path suppressed migrated duplicates without declaring WHICH
incoming indices it suppressed. replay.mjs keys two gates on that
declaration — safetyViolation() filters the indices out of the input side
before comparing lengths, conservationViolations() accepts a missing unit
only when it is part of a declared suppression — so both reported a
designed behaviour as corruption. Replaying capture s-77fe2779
(conversation e7394e05, request 11:41:05.778Z) under the serving gate
set: 1 safety violation (length 124 -> 123) + 1 conservation violation
(lost in[98]) before, 0 and 0 after, forwarded bytes unchanged. 059aae3
added the suppression at 13:35Z, after the day's 07:52Z sweep, so no gate
run had exercised it yet and the next daily sweep would have gone red. It
also restores the per-suppression event lines the "rule out ourselves"
attribution sweep reads, on the ~1-in-3 requests that reset.

The dispatched EXTENDED-class absorb is NOT built, and the directive,
BACKLOG entry and matrix row 4 now carry why. Its premise was that the
EXTENDED remainder is new harness text; measured, it is a standalone
system message the predecessor already carried, swallowed into the
migrated reminder (9 of 9 occurrences corpus-wide, 0 new text). Its
action was to re-emit that remainder at a frozen tail index; measured on
the real pipeline, the first forwarded divergence stays at 100, while
restoring the swallowed message at ITS index moves it to 123 of 124. That
un-merge is unit 2 of flap-move-mitigation-and-fidelity-gate.md, already
built on wt/fidelity/opus and blocked on the identity decision — so the
item is a duplicate that closes by merging there, not a second mechanism.

Evidence: docs/code-reviews/extended-absorb-report.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 5c4d70a, code + test paths only; the BACKLOG / threat-matrix / directive halves are fork-internal and stay behind)
…and moves surviving resets) reconciled with the reset-path duplicate suppression

Cherry-picks of aef760b and dc8c475 from branch wt/fidelity/opus (author
Gunther Schulz, Co-Authored-By Claude Opus 4.8 on both), squashed into one
integration commit because the second does not stand alone on current main.
Their content, verbatim except for the reconciliation below:

  aef760b  a recognized reminder move serves its first-seen form — no reset,
           no lost bytes. Cross-message join recognition (findJoinMoves),
           in-place substitution of the absorbed entry into the merged
           message's slot, `moved`/`reserves`/`suppressions{kind:"join-move"}`
           telemetry, and replay.mjs's wireRemovedIndices so a substitution is
           not mistaken for a removal.
  dc8c475  a reset keeps recognized moves — same argument, same riders as the
           pins it already keeps. findJoinMoves runs inside resetKeepingPins
           after the pin substitutions, on the same array.

RECONCILIATION with 5c4d70a (reset-path duplicate suppression, which landed on
main after dc8c475 was written and touched the same return statement). Both
suppression kinds now flow through the ONE `suppressions` array the gates read,
exactly as the success path already does:

  - the merged slot is declared `kind: "join-move"` and is NOT added to the
    removal set — the slot survives, carrying the re-served first-seen bytes;
  - a plain duplicate is declared without a kind and IS removed from the
    forwarded array;
  - `suppressed` counts the declaration array, matching the success path;
  - the reset's canonical is built from the KEPT entries (duplicates excluded,
    the success path's invariant) with a moved slot filing the ABSORBED entry
    rather than a fresh identity built from the merge (dc8c475's invariant);
  - the `messages` guard is `applied > 0 || moves.length > 0 ||
    suppressedR.size > 0` — the union of both features' guards.

Neither feature's behaviour is narrowed: the move branch is consulted before
the duplicate predicate, so a merged slot can never be routed down the removal
path, and a duplicate is untouched by the move machinery.

Baseline after reconciliation, three targeted files (join-move, normalization,
merge-suppression): 78 tests, 75 pass, 0 fail, 3 todo — the three TODOs are
dc8c475's, still todo, as expected before the identity fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit a1170a7; its tools/replay.mjs half arrived with the pr/verification-tools merge)
…pace, so one more copy of its text cannot re-bind it

Unit 2b was built and bite-proven and did NOT close the regression it was
written for, because the mechanism it blamed was not the one firing. Measured
on capture s-dc3f8071 (fixture reset-move-s-dc3f8071-196-197.json): a
recognized move keeps an entry alive in OUR canonical while CC has stopped
sending it, and that entry's key is (content-hash, role, occurrence-ordinal-
within-the-request). An ordinal is a claim about CC's array — an array the
entry is not in. The claim went false at n=197, when CC sent an eighth copy of
a recurring tail reminder: the copy took o=7, the entry bound to it 13 slots
away, and two things broke at once — the entry left `droppedNow` so no move
recognition could fire, and the inverted pair tripped `not-subsequence`. Same
shape, same merged-content hash, again at n=399->400.

So a re-served entry's identity is now its stored first-seen bytes plus the
canonical slot where we last forwarded them, marked `rs: true` at the mint, and
it does not participate in (hash, role, ordinal) wire matching AT ALL — not
looked up, not counted as dropped, not read by the canonical-order check. A
fresh copy of the same text takes the next free ordinal, matches nothing, and
classifies as an ordinary new entry on the existing append/splice path. Every
non-reserved entry keeps absolute matching byte-for-byte: the general ordinal
instability of duplicate copies under middle-copy drops is a pre-existing
class and deliberately out of scope.

What replaces the match is a per-request disposition over the entry's
neighbourhood, resolved exactly as findJoinMoves' condition (d) resolves it,
one of three in this order: RE-FIRE (the merged form is on the wire again ->
re-serve the stored bytes into that slot, declare the join-move, stay
reserved), RECLAIM (CC flipped back to the original form -> clear `rs`, bind
the entry to that wire index as an ordinary matched entry and rewrite its
stored key from that message's incoming identity), LAPSE (neighbourhood
resolvable, neither form present -> not carried into the rebuilt canonical).
Bounds unresolvable or crossed: nothing happens at all — no substitution and
no state change, raw forward. The lapse rule is the mitigation for the one new
risk the design introduces, re-serving stored bytes into a context CC has
pruned away, and it fails closed whenever its preconditions are not
byte-established on the current wire. Lifecycle is self-limiting by
construction: an `rs` entry persists exactly as long as its re-serve fires.

findJoinMoves gains condition (f) — merged wire message role "system" and
absorbed entry stored role "system" — closing the latent role gap the unit-2b
report surfaced (§c5). The same constraint binds both disposition probes.

Red-first, ten mutations each biting the bite that names its condition: match
exclusion, the mint's `rs`, re-fire, reclaim, lapse, fail-closed bounds, (f) at
the mint on both sides, (f) on the probes, and the canonical-order exclusion.
The reclaim bite's first draft survived its mutation — a lapse followed by a
fresh entry with the same bytes was observationally identical in that shape —
and now asserts the property that actually separates them: the reclaimed
message is a MATCHED entry, so it is not an insertion and the request is a
plain tail append.

Two of unit 2b's three TODO tests flip to passing unedited, including the
five-gate one: the fixture's stability violation at n=197 is gone. The third
does not, and its expectation is not edited — its control asserts
`action === "reset"` at n=197, and with the re-bind removed there is no
inversion, so n=197 classifies `normalized`. The reset was the symptom the
unit-2b report named as such; the criterion that test carries (stability
quiet) is met by the sibling that asserts it directly.

Threat-matrix row 3, restart declaration: no key-scheme change. (h, r, o)
storage is unchanged and `rs` is a new OPTIONAL field, so canon files written
by the old code contain no `rs` entries and take identical decisions under the
new code — measured, not assumed, by tools/verdict-ab.mjs --seed-from-a.
Production has never run unit 2, so no deployed canon file can carry a
re-serve. The restart is cache-transparent for every existing conversation;
the new machinery activates only at the first post-restart move recognition.

Directive: docs/directives/reserved-entry-identity-directive.md

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit fad6f6b)
…t a scan over `matched`

Self-caught while re-reading the previous commit: the reset path's canonical
rebuild resolved "was this wire index reclaimed?" with a `matched.find` inside
a `.map` over every kept entry — O(entries x matched) on a path that runs on
roughly one request in three (this file's own measurement: 125 resets across
350 requests), against requests that reach 2000+ messages on the large
captures. The disposition pass already knows the answer, so it records it:
wire index -> ci, one map lookup at the rebuild.

Pure refactor of one expression, no decision changes — proven by
`tools/verdict-ab.mjs` against the parent commit rather than argued, and the
ten mutations of the parent all still bite.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 0cc05c7)
…e definition, all three unit-2b todos retired green

The control asserted action==='reset', the exact signature the
reserved-entry identity build removes; verifier 1's definition
(n=197 normalized, re-fire continues, bytes at the merged slot hold)
is what the test now pins. Its red against the pre-build tree is on
record in the A/B (actual normalized vs expected reset). TODOs 16/17
flipped unedited and lose their todo markers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0171Mpvi9GsSnJjBNnbfxU2u
(cherry picked from commit 8e3c265)
…the mint guaranteed (insertion slice of fork 9983a1b)

The two conditions read as defensive padding for an impossible case. They
are not: the mint happened in a PREVIOUS process and the entry arrived
through a canon file on disk, so a deserialization boundary sits between
the guarantee and this check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 9983a1b, code path only; the report and directive are fork-internal)
…ipt-shape fixture

Their identifiers, committed upstream, public in the upstream tree
before this scan existed — a pre-existing-third-party file the scan
must name, not go red on forever. Provenance beside the entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016y33RMV399iYMXFEbAfQCk
Gunther-Schulz added a commit to Gunther-Schulz/claude-code-cache-fix that referenced this pull request Aug 1, 2026
…ified, pushed; draft cnighswonger#295 open

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016y33RMV399iYMXFEbAfQCk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant